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Teaching how to Listen. Blended Learning for the Development and 
Assessment of Listening Skills in a Second Language 


Abstract 

This paper discusses the integration and effectiveness of blended learning for the development and assessment 
of listening skills in a second language. The development of oral abilities (listening and speaking) is one of the 
most challenging and neglected aspects of second language learning (Vandergrift & Goh 2012, Graham & 
Santos 2015). Listening comprehension work in particular is crucial in the early stages of second language 
acquisition, and, therefore, for ab-initio language students, for whom processing and decoding auditory input 
can be very challenging. In 2014 a set of online listening quizzes was created and integrated into two ab-initio 
Italian courses. The aim was to offer engaging, flexible listening comprehension practice and assessment, 
which would extend the students’ learning experience, stimulate their learning motivation and allow for a 
better use of face-to-face teaching in the classroom environment. Having conceptualised listening as a process 
rather than a product we designed tasks to teach learners how to listen, rather than merely test their 
comprehension. The validity of the quizzes as a means for the development of listening skills and as a tool for 
formative and summative assessment was subjected to systematic analysis via an online student survey. The 
large amount of data collected reveals that the quizzes were a key element in the development of listening 
skills and the delivery mode did not only meet the students’ learning needs but it was clearly preferred to in- 
class assessment. 
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Introduction 

This article brings to the fore the need to reconceptualise the teaching and assessment of listening- 
comprehension (LC) skills in second-language acquisition, in a context of demand for more- 
efficient teaching practices and more online engagement. The research undertaken for this project 
involved the creation and evaluation of online LC quizzes for two different courses of Italian taught 
at the University of Western Australia. To understand the rationale of this investigation, it is 
necessary to begin by presenting the institutional context in which it took place. 

The University of Western Australia is located in urban Perth and is a research-intensive 
university. It is part of the Australian “Group of Eight”, Australia’s eight leading research 
universities. In 2012 the University of Western Australia introduced a restructuring of 
undergraduate degrees known as New Courses. Under these changes students are required to take 
at least four subjects providing knowledge beyond their chosen field of specialisation and offered 
in a different faculty to their own. For example, a student studying a Bachelor of Commerce may 
take four courses from the Faculty of Arts. This new structure resulted in unprecedented levels of 
enrolments in language subjects. Unlike previous years, when students studying languages mainly 
came from the Faculty of Arts, they now come from all faculties across the university, bringing 
different needs and learning styles. 1 

The boosting of blended learning through the integration of online listening quizzes was therefore 
an important way to help us meet two main challenges. First, we wanted to help maintain the 
growth in enrolments by offering students engaging, flexible LC practice. Within the field of 
second-language acquisition (SLA), the development of oral abilities (listening and speaking) is one 
of the most challenging and neglected aspects (Vandergrift & Goh 2012; Graham & Santos 2015). 
LC was targeted because of its impact on success in language learning: “In particular, the use of 
listening strategies can make authentic texts more accessible in the early stages of learning a 
language, so that the process becomes more relevant and interesting for the learners” (Vandergrift 
1999, p. 174). Second, our project was an attempt to increase efficiency in assessment; this would 
take place outside of the classroom, allowing for a better use of face-to-face teaching in the 
classroom environment. Since little attention has been devoted to university students’ perception of 
online listening, particularly in Italian L2, our research also aims to address a gap in the literature. 

This article is structured as follows: Section 2 reviews some relevant issues in LC research and 
pedagogy, which provided the theoretical background for our investigation. Section 3 describes the 
project and exemplifies some of the materials developed. This is followed by the project evaluation 
(Section 4), and the data discussion (Section 5). Conclusions are drawn in the last section. 


2. Listening-comprehension pedagogy 

Our research stems from the most recent discoveries in LC, which show that it is “a key component 
of language acquisition and an important foundation for success in language... programs” 
(Vandergrift & Baker 2015, p. 391). Particularly, we conceptualised listening as a process rather 
than a product (Graham & Santos 2015), and we attempted to teach learners strategies on how to 
listen, rather than merely test their comprehension (Thompson & Rubin 1996). Krashen understood 


1 These changes have been documented extensively for Italian in Caruso and Brown (2015), and for all languages in Brown 
and Caruso (2016). 
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LC’s important role in foreign-language acquisition early on in the history of research on SLA. 
Following his first claims that “comprehension may be at the heart of the language acquisition 
process” (Krashen 1981, p. 102), he went on to demonstrate that it is precisely in the moment when 
we comprehend that we begin to learn. In other words, we learn through comprehending: “we 
acquire language when we understand messages” (Krashen 2013, p. 3). 

Despite Krashen’s early discoveries, listening has long been treated as the “Cinderella of 
communication strategies” (Vandergrift 1997, p. 494) or “of the four macro-skills” (Flowerdew & 
Miller 2005, p. xi), and there is still little research devoted to the study of LC and the development 
of the skill of listening in a second language (Field 2012). Absalom and Rizzi (2008, p. 56) note a 
“lack of research which explores the effects of research online listening may have on L2 
proficiency’. As a consequence, a gap has developed “between theoretical development in listening 
research and materials for listening pedagogy” (Graham & Santos 2015, p. 96), with repercussions 
for the learners, who are not instructed efficiently on how to approach listening. 

One of the main problems is that listening comprehension has long been focused solely on testing 
students’ ability to listen to and comprehend audio tracks: “Too often teachers only use listening 
activities to test the listening abilities of their students, which leads to anxiety and apprehension. 
This is not a context favourable to the acquisition of useful listening strategies” (Vandergrift 1999, 
p. 174). Earlier discussions on LC were focused on the classification and grading of listening tasks 
in terms of difficulty (Fish 1981; Nunan 1989; Richards 1983; Ur 1984). Furthermore, the 
“perception of difficulty resided in the difficulty of the material used as the content for the 
comprehension activities, and was borrowed from readability measures for written texts” (Hoven 
1999, p. 88). While listening and reading share receptive language processing, listening needs to be 
studied independently from reading (Vandergrift & Baker 2015, p. 392). 

Hoven (1999, p. 3) emphasises how, “as changes in the focus of language teaching and learning 
have moved from content- or teacher-centred to more learner or learning-centred approaches, the 
focus in listening comprehension has also shifted”. More importance is now given to listening in the 
learning of a foreign language, as demonstrated by recent studies (e.g. Graham & Santos 2015; Lee 
& Lee 2012), and to the social and cultural impact listening can have on students. 

Lieske’s (2007) contributions are particularly relevant for selecting audio texts and designing tasks. 
Lieske identifies five elements of effective listening materials: content validity, purposefulness and 
transferability, retrieval of information from long-term memory, teaching new listening skills and 
authenticity. With respect to transferability, she notes that 

an activity that asks the listener to count the number of times s/he hears the word go lacks 
transferability and purposefulness because at the content level it does not reflect normal, natural 
behaviour. In addition, the type of information that the learner must provide (i.e., the outcome of the 
task) is not necessary to accomplish a listening task in the real world. Finally, while this activity may 
develop the ability to perform well on classroom tests, it does not help the learner master the listening 
skills that are required to have real-world conversations (Lieske 2007, p. 41). 

Richards (1983, p. 171) cautions against focusing on the “retrieval of information from long-term 
memory rather than on the processing activities themselves”. If the listener is asked to recall 
information after the listening passage is over, the focus is on memory instead of comprehension. In 
addition, question-oriented instruction, such as listening activities that use true-false questions and 
follow-up vocabulary exercises, does not require the learner to use the language functionally 
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(Morley 1991). Similarly, Lieske emphasises that the LC activity needs to teach students “skills that 
enable them to listen in real-world contexts” (Lieske 2007, p. 42). 

Authenticity is the last element listed by Lieske when designing LC. It is essential for transferability 
to the real world (Porter & Roberts 1987); unauthentic dialogues do not prepare learners for 
realistic communicative events. Artificially slow dialogues lack real-world lexis, avoid reduced 
forms and have no hesitations or rephrasing. As a consequence, they may be too simplified and too 
easy for a learner to understand. Dialogues also lack authenticity when the speakers exaggerate 
their intonation, when there is unnatural repetition and when participants say equal amounts (Porter 
& Roberts 1987). In contrast, dialogues that have conversational overlap, background noise and 
attention signals (e.g., uh-huh, mmm) reflect real-life conversations, making them more authentic 
(Lieske 2007). Videos can also provide excellent authentic materials for LC. 

Videos have long been used in the teaching of foreign languages, and particularly in the teaching 
and testing of LC. According to Wagner, this is due to the belief that “including the non-verbal 
components of a spoken text will be useful for listeners in comprehending the aural input” (Wagner 
2010, p. 493). Communication occurs on different levels, through a multitude of codes that imply 
paralinguistic, extra-linguistic and proxemic elements. Audiovisuals are, thus, a pluridimensional 
reality that can be exploited on different syntactic levels in the foreign-language classroom (Forgacs 
et al. 2005, p. 173). Moreover, the use of audiovisual materials in the foreign-language classroom 
offers students direct contact with the culture represented on screen. Porta (2013, p. 90) emphasises 
the “inherent cultural value” that videos found online can have for language students, where culture 
is defined not as the “‘big C’ kinds of culture... complex, institutionalized and historically 
differentiated”, but rather the “‘little c’ culture, the everyday, tangible, subjective, even routine 
aspects of L2 culture” (p. 93). 

Assessment is now increasingly taking place online rather than in the classroom. Computer 
technology has provided an alternative learning space, a different delivery mode to that of the 
classroom. It provides for blended teaching and learning based on alternating online and offline 
activities and materials (Tomlinson & Whittaker 2013). Lee and Lee (2012) specifically addressed 
students’ preferences for online versus offline listening activities in English as a second language at 
university level. They found that “an optimal design and the successful implementation of the 
blended learning model in listening instruction require the teacher to play a crucial role as a 
designer, a selector of effective activities, materials and multimedia tools, and a monitor who 
provides timely scaffolding” (p. 1). Our project also addresses such implications. 


3. Project description 

Experimentation with listening quizzes in Italian courses started in semester 1 of 2014, when 
students from the ab initio stream were required to take three online quizzes to practise listening 
skills. These tasks were integrated into their course but did not count towards their final mark. Since 
the feedback received at the end of that semester was very positive, it was decided to design and 
critically evaluate a full set of listening quizzes that would serve as a tool for formative and 
summative assessment. 2 The criteria that were followed in the selection and integration of the 
materials to be designed were grounded in the research presented above: 

- input quality 


2 The project was supported by a UWA Improving Students Learning grant obtained in 2014. 
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- consistency with in-class activities/content 

- progression. 

The project was targeted to our two largest cohorts: the ab initio semester course ITAL1401 (118 
students in semester 2 of 2014) and the semester course ITAL1402 (81 students in semester 2 of 
2014). 3 A total of 33 quizzes were created between July and September 2014 based on audio and 
video tracks that satisfied the three main criteria above, and were “meant as continuous and 
repeating cycles of activities” (Green 2013, pp. 25-26) rather than stand-alone products. The 
quizzes included mainly multiple-choice questions, with the possibility of question preview, to 
encourage the activation of prior knowledge through key vocabulary. Chung (2002, quoted in 
Vandergrift 2007, p. 198) found that “multiple choice questions had a greater influence on listening 
success than open-ended questions”. 

Both audio tracks and audio-visuals were implemented in the program. The audio and video files 
(between two and four minutes long for the audio files, up to five minutes for the video files) were 
sourced amongst textbook materials or online (Youtube, www.edilingua.it, 
www.guerraedizioni.com). One of the videos was filmed by one of the authors of this article. 

Videos were used in the context of our project as an aid to listening comprehension, specifically 
when moving away from mere word- and sound-discrimination, when introducing more difficult, 
faster conversations, to ensure progression, and for our students to be exposed to Italian culture as 
well as the Italian language. The use of authentic, unscripted audio-visuals in our project provided 
“direct contact not only with the language, but also with the culture represented on the screen” 
(Forgacs et al. 2005, p. 173), and the learning-management system platform became the 
environment for a cross-cultural encounter. The pluridimensionality of the audio-visual message 
emphasised by Forgacs et al. was thus exploited on different didactic levels. 

Students accessed the quizzes via their learning-management system page, based on Moodle. The 
University of Western Australia Multimedia Centre team provided technical advice and support on 
the interface between sound files and quizzes. It was decided to incorporate 10 assessable quizzes in 
each of the two courses: one quiz per week from week 3 to week 12 (a one-semester subject 
consists of 13 weeks). Altogether they counted 6% towards the total assessment for the course. For 
the first eight quizzes (specifically designed for formative assessment), students could listen to the 
track and attempt the quiz as many times as they wanted, to encourage the planning and verification 
stages of Vandergrift’s (1999) pedagogical cycle. Students had total control over the audio tracks, 
which they could pause and play as necessary. The audio files were made available at the start of 
the week, with the quiz, but while the audio files were accessible up until the end of the course, the 
quiz closed at the end of the week. These eight quizzes were weighted 3% in total. For the last two 
quizzes, students could only listen to the audio track twice, in non-stop listening mode. They only 
had two attempts at the quiz (with the final mark set as the highest mark), and had a time limit to 
complete the task. 4 These two final quizzes were worth the remaining 3%. At the same time, 
students were also expected to complete one extra listening quiz per week, which did not count 
towards the assessment but was aimed at offering formative feedback. 5 Quizzes provided students 


3 In these courses successful students achieve, respectively, Level Al and Level A2 of the European Framework of 
Reference for Languages. 

4 ITAL1401 two final quizzes had a time limit of eight minutes. ITAL1402 quizzes had time limits of nine and 17 minutes 
respectively. 

5 Besides the listening quizzes, weighted 6% of the final mark, the other assessment items for both ITAL1401 and 
ITAL1402 were: four in-class tests (24%), three online activities (10%), role-play (video-recorded and uploaded to 
YouTube. 10%), participation (15%) and written examination (35%). 
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with immediate final scores and included, in different degrees, explicit feedback in the incorrect- 
answer field, to guide students with the listening process 6 . 

The section that follows addresses and illustrates the progression of the quizzes. The first quizzes 
created for ITAL1401 were aimed at practising mainly bottom-up aspects of listening, such as 
sound and word discrimination. The audio tracks were based on content that had been covered in 
class, and were fairly simple, while still displaying the characteristics of real-life conversation. 
Figure 1 illustrates the type of questions created for these early quizzes. 


A: 11 suo nome? 

Question 1 Di doVd il signor Lopez? 

B: Mariano Lopez. 

Not yet answered 

A: Lopez? Come si scrive? 

Marked out of 2 .a Select one: 

B: LOPEZ. 

.... „ t . O a. 6 spagnolo, di Madrid. 

V Flag question 

A: Bene, Signor Lopez, di dov’e? 

o Edic auestion O t. 6 spagnolo, di Barcellona. 

B: Sono spagnolo, di Barcellona. 

O c. d spagnolo, di Tarragiona. 


Figure 1. Excerpt of transcript and quiz question. Focus on word recognition 

A: Your name? B: Mariano Lopez-A: Lopez? How do you spell it? B: LOPEZ-A: Good, where 

are you from Mr Lopez? B: I’m Spanish, from Barcelona 

Question 1. Where is Mr Lopez from? Select one: a) He’s Spanish, from Madrid; b) He’s Spanish, 
from Barcelona; d) He’s Spanish, from Tarragona 

From tasks based on sound and word discrimination/recognition, students progressed to complete 
quizzes that used tracks with “extension” content: vocabulary or structures extending topics covered 
in class, or requiring decoding and/or inferencing. This was aimed at emphasising comprehension. 
Videos were also used as listening texts, particularly to support authenticity and, following Field 
(2008), to introduce variety in the range of tasks. The paralinguistic features of videos (kinesics, 
proxemics, prosody) allowed us to offer students fast-paced conversations that reflected real-life 
communication. From week 7 (midway through the course), the focus of the tasks was significantly 
more on comprehension and inference rather than recognition, as shown in Figure 2: 



Question 5 Secondo te, i due personaggi: 

Che gelato vuole? 

Not yot answered 


Marked out of 2 00 Select one 

Le faccio subito il caffe. 

„ O * sono amici 

Y Flag question 


Edit question O b *» conoscono abbastanza bene 


O c. non si conoscono molto bene 


Figure 2. Excerpt of transcript and quiz question 5. Drawing on prior knowledge 

What ice-cream would you like, Sir? I will make your coffee immediately, Sir. 

Do you think the two characters: a) are friends; b) know each other fairly well; c) do not know each other very 
well? 


This example shows that students had to identify the formal form of address in Italian, which is 
encoded in the verb ending ( vuole , “you want”, formal) and the pronoun ( Le , “to you”, formal), to 
be able to determine whether the interlocutors were friends or acquaintances. This reflects the idea 


6 For example, where the correct answer was “e spagnolo, di Barcellona” (he is Spanish, from Barcelona), but the student 
selected “e spagnolo, di Tarragona”, the feedback for the incorrect answer was, “Listen again - the name of the city ends in 
-ona, but is not Tarragona.” 


5 








Journal of University Teaching & Learning Practice, Vol. 14 [201&], Iss. 1, Art. 14 


that “listening involves attending to contextual features such as the setting (where the listening 
event occurs), the people involved and their relationship, the purpose of the listening event, its 
degree of formality” (Graham & Santos 2015, p. 13). 

Finally, in week 10, we introduced authentic unscripted videos, such as one filmed by one of the 
authors in Italy, featuring a customer ordering food at a cafe. Together with the features listed 
above, these videos represented an extra challenge for our students, as the speakers used their own 
regional accents and their authentic speaking pace. By watching and listening to these videos, our 
students were virtually catapulted into contemporary Italy, experiencing a cross-cultural encounter 
via the unit’s learning-management system. 

The last two quizzes provided a further challenge in that students could only listen to the audio 
track twice and had time limits. The technical interface was simple, with the audio tracks embedded 
into the quiz. 


Quiz navigation 

□□□□□a 

00000 

Finish attempt ... 

Time left 0:07:50 


ITAL1401 Italian Studies 1 - SEM-2 (CRAWLEY), 2014 

You can preview this quiz, but if this were a real attempt, you would be blocked because: 

This quiz is not currently available 


[ Start a new preview ) information 

IP Rag question 

41 Edit question 

Navigation 

My LMS Home 
Site home 
O Site pages 
O My profile 
O Current unit 


O click to play - this audio 
file will only play once 


O click to play - this audio 
file will only play once 


O ITAL1401 SEM-2 2014 


O Participants 
O Badges 
O Torw-nmi,™, 


Question 1 

Not yet answered 
Marked out of 1.0 


La casa di Lucia 6: 
Select one: 


Figure 3. Embedding of the audio track 

Students of the ITAL1402 stream, who had studied ITAL1401 in semester 1, were given only a few 
quizzes featuring sound and word discrimination, with most of them based on comprehension. 


4. Project evaluation: Research methodology and procedures 

To investigate the effect of online quizzes for the development and assessment of listening skills we 
adopted a quantitative approach and used an anonymous survey as our research tool. The survey 
was designed with Qualtrics (https://www.qualtrics.com) and comprised 23 multiple-choice 
questions (five with the possibility of leaving comments), as well as the final question “Do you 
have any comments about the quizzes?” 

The final question allowed us to collect a useful amount of qualitative data and gain further insights 
into the relevance of the quizzes. Bourque (2003, pp. 111-112) refers to such open-ended questions 
as “ventilation questions”, as they “allow respondents to ventilate their feelings about the topic or 
the questionnaire”. The multiple-choice questions were formulated mainly around the following 
topics: a) completion of quizzes; b) perception of online assessment; and c) impact on learning. 
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Although research on the use of quizzes and other e-tivities in SLA indicate that quizzes typically 
have a good rate of completion (Baccari 2015; Edirisingha et al. 2007; Moyer 2006), we chose to 
investigate whether and how often students would access our materials, to be able to create cross¬ 
tabulations with other variables. Most importantly, we wanted to verify how students found the 
listening texts and the quizzes in terms of content and progression, and whether they believed that 
the quizzes helped them with their listening skills and the general learning of Italian. 

The survey was distributed at the end of the subject, in October 2014, via an email sent to all 
students from ITAL1401 and ITAL1402. The survey could only be taken once. Ninety surveys 
were submitted, corresponding to 45% of the total students. In addition, a total of 37 comments 
were elicited through the final survey question. A further 38 comments were added in some of the 
multiple-choice questions. Some of these comments are quoted in our discussion. 

In the next section we consider the validity of the materials as perceived by the students. For each 
of the points below, no significant difference was identified between the groups from ITAL1401 
and ITAL1402. 


5. Results and discussion 

5.1 Completion of quizzes 

Quizzes for assessment 

Seventy percent of students who participated in this survey stated that they had completed all 10 
quizzes, and 26% stated they had completed most of the quizzes. Only 4% of the sample completed 
half or fewer than half of the quizzes. When asked to choose the principal factor for not attempting 
at least one of the quizzes (Figure 4), the overwhelming majority of students (78%) put it down to 
forgetting. 7 


If you didn’t submit all online quizzes, 
what was your main reason why not? 


100% 

80% 

60% 

40% 

20% 

nnz 

78% 






AO/ 7% 11% 


4% ' /0 q% 

U7U 

I forgot It was too I was too I didn't think Other 

difficult busy it was 

worthwhile 


Figure 4. Reasons for not completing the quizzes 


7 One student suggested that students be given “more freedom in completing them; e.g. make five available at beginning of 
semester and then the rest after mid semester break, this would eliminate the problem of forgetting to submit each week”. 
While this may be a valid solution for the issue of forgetting to submit quizzes, it does compromise the progressive nature 
of the quizzes, and therefore, their formative value. 
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Busyness (8%), the difficulty of the quizzes (4%) or some unspecified reason (11%) were among 
the other factors indicated by students. No student selected “it wasn’t worthwhile”. 

A further positive statistic emerged from the number of times students attempted the quizzes and 
listened to the audio/visual materials. While 27% attempted the quizzes once, 67% took advantage 
of the possibility of attempting the quizzes two or more times, and 6% selected ‘more than three 
times”. Furthermore, when asked to indicate whether they had listened to the audio-visual material 
more than once, an overwhelming 93% answered “yes”. Although students were not asked why they 
had listened to the audio/visual material more than once, this datum seems to affirm the formative 
value of the quizzes and students’ willingness to spend more time to improve their listening skills 
and, of course, to better their marks. Additionally, students were asked whether they had used the 
audio-visual materials for any private study activities (e.g. dictation, further comprehension, 
vocabulary building). Forty-four percent selected “yes” (with the remaining selecting “no”), further 
supporting the formative value of these tasks. 

Quizzes not for assessment 

In keeping with a formative approach, students were also offered non-assessed quizzes that did not 
count towards their final mark. Almost 60% of the students had attempted one or more of these 
quizzes (Figure 5). 


Did you attempt the extra (non-assessed) 
quizzes? 

50% 

40% 

30% 

20 % 

10 % 

0 % 

Figure 5. Completion of quizzes not for assessment 



Students who attempted these non-assessed quizzes indicated via a different survey question that 
the principal reason for doing so was “to develop their listening skills” (66%). Students were also 
aware of the broader benefits for their Italian (20%), and a small percentage attempted these quizzes 
primarily for their enjoyment, selecting “because they are fun” (10%); 4% (two students) chose 
“other” and left comments. One student commented that they would use them in preparation for the 
final two quizzes, and the other that they would use them for general vocabulary learning in 
preparation for the written exam. It can be concluded that more than a few students availed 
themselves of these non-assessed quizzes and found them to be a useful tool for improving their 
listening competence. Overall, these findings confirm that our aim of offering students extensive L2 
input to facilitate effective learning (Ellis 2005) was met. 


5.2 Student perception of the materials 
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Student satisfaction 

In order to evaluate student satisfaction, students were asked to respond to the question: “Did you 
enjoy doing the quizzes?” Sixty-six percent of students replied “yes”, and only 6% replied “no”. 
Given that this was an assessment (and not a movie for personal enjoyment), this is a very positive 
outcome, particularly in light of the association between student enjoyment/motivation and 
learning success (Field 2012; Grabe 2009; Vandergrift 2005). We believe that the fact that the 
quizzes contributed only slightly (6%) to the student's final mark helped them cope with the anxiety 
often associated with listening, and to develop confidence and a positive attitude about listening (cf. 
Vandergrift 2007). 


Furthermore, the survey revealed that most students (66%) favourably received the online platform 
for these quizzes, with only 18% preferring in-class assessment (Figure 6). 


Do you think that online assessment for 
listening is better than in-class assessment? 



Figure 6. Online vs. in-class assessment 

Students’ perceptions about assessment have a significant influence on how students approach 
learning (Struyven et al. 2005), and these results confirm previous studies on the important place 
that computer-aided/online tasks have in the language class (Lee & Lee 2012; Van Patten et al. 
2015). Those students who preferred the online platform were asked what they believed were the 
best aspects of online quizzes; the majority indicated that the possibility of listening to the quizzes 
when, where and as many times as they liked was the principal reason they preferred online 
quizzes, expressing an appreciation for autonomous learning. 
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80% 

70% 

60% 

50% 

40% 

30% 

20 % 

10 % 

0 % 


What is the best aspect of online quizzes? 

75% 


7% 

-A- Lu / U 

7% 


^^B 


^^B 


Other 

It saves class 
work time 

It makes learning 
more fun 

Being able to 
listen to it when, 
where and as 
many times as I 
like 


Figure 7. Students’ perception of online quizzes 


One student commented, “I like the fact that I was in control of playing the audio - in class there is 
always a chance of extraneous noise.” This is a positive result if it is considered that “listening is 
perceived by many listeners in different contexts as the skills in which they have the least control” 
(Graham & Santos 2015, p. 17). Several students who selected “other” listed factors such as instant 
feedback, and decreased pressure, including the impression that these quizzes felt less “like a test 
but that you are just doing an activity” (student’s comment). The latter point suggests that online 
listening can be effective in preventing the impact of anxiety on listening proficiency (Mills et al. 
2006). This positive response to the online mode of delivery is certainly an important result 
supporting the efficacy and relevance of blended learning in the context of language learning and 
assessment. 


Those students who replied negatively to the question “Do you think that online assessment for 
listening is better than in-class assessment?” were also asked to justify their choice (Figure 8). 


Why did you think online assessment is 
worse than in-class assessment? 



Other Forgetting to do I had trouble Another extra 
the assessment accessing the thing to do at 
quizzes home 


Figure 8. Justifying preference for in-class assessment 


Seven students selected forgetting to do the assessment, having trouble accessing the quizzes and 
the addition of another thing to do outside of class; however, it should be noted that nine chose 
“other”, of whom seven elected to leave comments in response to this question. Of the relevant 
comments, three students mentioned that the quizzes were not as challenging given the number of 
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attempts that were permitted; however, two of these went on to say that they were grateful they 
could get good marks for the quizzes as a consequence (cf. student perceptions on the level of 
difficulty below). One student simply stated that they were “not as effective”’ without further 
explanation. Cross-tabulations reveal that five of the seven students who did not enjoy the quizzes 
also did not believe that the quizzes improved their listening skills, which further supports the 
correlation between motivation and proficiency (Vandergrift 2007; Graham & Santos 2015). 

The impact of technology 

Question 12 of the survey asked students if they had encountered technical difficulty with carrying 
out the quizzes. The largest majority (75%) answered negatively, with the remaining 22 students 
(25%) admitting to have had some issues (Figure 9). 


Did you have technical difficulties with 
the online quizzes? 



No, 75% 

Figure 9. Technical difficulties 

All 22 students who did encounter technical problems elected to leave some comments. Several 
mentioned their trouble with buffering/loading issues whilst attempting the final two quizzes. This 
problem may be a result of the embedded audio player that was chosen for the final two quizzes. An 
alternative audio player for these final two quizzes may need to be considered to potentially avoid 
these buffering issues in the future. Educating students about the importance of a strong internet 
connection prior to starting accessing the online materials would also lower the number of technical 
problems. 

To further investigate the effect of technology on the students’ perceptions of the online activities, a 
correlation was drawn between the figures on technical issues and the perceived usefulness of the 
activities. This data (Figure 10) suggests that students’ perceptions of the activities’ usefulness was 
unrelated to whether the students had encountered technical problems. Given the small sample of 
answers, however (only 22 students encountered technical issues), final conclusions must be made 
with caution. 
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no technical difficulties 
■ technical difficulties 


Figure 10. Technical difficulties and perceived usefulness of quizzes 
Student perceptions of the level of difficulty 

Contrary to those few comments left by students who indicated that the quizzes were not as 
challenging, when all students were asked to respond to the question, “How did you find the 
language used in the dialogues/videos?”, most indicated it was challenging (47%) or right for the 
competence expected (44%). Only 6% of students stated that the quizzes were too easy, and on the 
other end of the spectrum only 4% stated that they were beyond their ability. These responses 
confirm that the level of difficulty of the carefully selected tracks was appropriate for the quizzes. 
Furthermore, the large number of responses indicating that these quizzes were challenging suggests 
that students were provided with a tool that could help them to improve their listening skills. One 
student commented, “The quizzes were challenging.... I really enjoyed the level of the listening 
comprehensions because I felt like they pushed me a lot.” This is in line with Absalom and Rizzi’s 
(2008, p. 62) findings: “The continuous engagement with the listening tasks as opposed to the text- 
based task confirms our hypothesis that the former are psychologically taxing on students which 
leads to higher level of motivation.” Several students who commented on the easiness of the 
quizzes referred not so much to the audio tracks as they did to the number of attempts they had been 
given. Although this was certainly not the view of the majority, a reduction in the number of 
attempts can be considered for future implementation, although care would have to be taken not to 
affect the formative value of the quizzes. Alternatively, other ways of grading the quizzes could be 
considered, such as basing the final mark on the averages of their attempts. 

Most students were aware that the level of difficulty increased throughout the semester: 80% agreed 
that the quizzes were progressively more challenging, 16% did not notice any change in difficulty 
and only 4% thought they did not become more difficult. These data largely confirm the formative 
value of the quizzes. It is possible that some students may have considered that their ability to 
achieve similar marks remained the same throughout the semester, and perhaps on this basis they 
did not believe the quizzes were progressively more challenging. 

As stated earlier, in preparing the quizzes, audiovisual materials were chosen and questions were 
formulated to reflect and reinforce topics covered in class, and the survey conducted for this study 
reveals that most students were aware of this: 71% stated that the quizzes were generally in line 
with the content covered in class, while 22% did not notice, and only 7% thought they were not. 
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Implications for further integration of blended learning 

As stated above, students indicated that by and large they preferred an online, rather than in-class, 
platform for assessment. Student responses were also largely positive to the question “Would you 
have liked to see other skills (reading comprehension, vocabulary, grammar) assessed online?” 
Sixty-two percent of students answered positively, 26% were neutral and only 12% answered 
negatively, with one student commenting, ’’For something like listening, which I find challenging, I 
would rather do that in class with the assistance of a teacher.” Furthermore, in response to another 
question asking whether the stuents would have liked to have additional non-assessed activities 
online, students generally showed interest: 22% selected “yes”, and 44% indicated that they “would 
probably use them”. Whilst 34% selected “no”, this feedback does support a general interest in 
blended learning. The responses to both of these questions suggest, therefore, that students would 
welcome additional online activities. On this matter one student commented, “I think a wider range 
of quizzes would be better for overall learning of the language. Instead of just restricting online 
assessments to listening, should [sic] incorporate a wider range such as grammar, translation and 
reading comprehensions.” This comment goes to the heart of the important question of how much 
work can be offered online versus what should be delivered in the classroom. Responses from this 
survey certainly suggest there is scope for a greater integration of online activities and assessment 
within these courses to meet students’ preferences. 


5.3 Impact on learning 

Listening skills 

The efficacy of these quizzes to develop listening was evaluated holistically by the question “Do 
you think that the activities helped you improve your listening skills?”, to which 92% of students 
responded positively and 8% negatively. 


Do you think that the activities helped 
you improve your listening skills? 

No, 8% 

a 


Yes, 92% 

Figure 11. The impact of quizzes on listening skills 

While it is not possible to draw conclusions about the nature of the quizzes validity on the basis of 
this dichotomous survey item, these raw percentages reveal that students did find them useful. The 
students’ comments are also mostly positive, with some displaying emotive language features 
(“definitely helpful”, “I can understand my Nonna now”, “incredibly useful”, “very helpful”. The 
negative comments point mainly to the students’ difficulty in engaging with the use of bottom-up 
strategies. The following three comments illustrate this point: “I found some quite challenging and 
fast paced which made it difficult to distinguish between words”; “I found them a bit too hard to be 


13 




Journal of University Teaching & Learning Practice, Vol 14 [201 &], Iss. 1, Art. 14 


able to learn anything from them. I think it was difficult to gain much from them due to this”; 
“One's [sic] with lengthy dialogues that you lose your spot in are kind of pointless because all you 
do is replay it again. I like to try and understand on the first listen.” 

Students were then asked to specify the areas in which they felt they had most improved as a result 
of attempting these quizzes. Table 2 reveals that students considered the quizzes to be most useful 
for learning to follow a conversation in Italian and the least useful for improving their accent. 



Answer 

1 

2 

3 

4 

5 

1 

Learning to 
distinguish words 

38.9% 

40.0% 

10.6% 

4.7% 

5.9% 

2 

Learning to focus 
on the sounds 

10.6% 

25.9% 

27.1% 

27.1% 

9.4% 

3 

Learning new 
words 

4.7% 

7.1% 

36.5% 

30.6% 

21.2% 

4 

Improving my 
accent 

3.5% 

7.1% 

4.7% 

30.6% 

54.1% 

5 

Being able to 
follow a 
conversation in 
Italian 

42.4% 

20.0% 

21.2% 

7.1% 

9.4% 


Table 2. Impact of quizzes on specific listening skills 

They indicated that the listening quizzes did help their sound and word discrimination (“learning to 
distinguish words” was selected as the second-most improved skill); one comment highlighted the 
usefulness of the quizzes in bottom-up aspects: “...it certainly helped to tune my ear in”. However, 
from these findings it can be concluded that students could see that listening comprehension implies 
being able to do more than distinguish words and recognise sounds, and far from being a mere tool 
for assessing listening skills, the quizzes helped them to improve their listening skills, as well as to 
progress in other areas of learning. 

Listening quizzes as opposed to other input sources 

Students were asked to select, from a list of items, which one they thought was the most helpful for 
improving their listening skills (Table 3). 



Answer 

1 

2 

3 

4 

1 

Exposure to your language tutors 

57.6% 

20% 

11.8% 

10.6% 

2 

Online listening quizzes 

27.1% 

38.8% 

25.9% 

8.2% 

3 

My own personal listening to 
Italian/exposure to Italians 
outside of class 

8.2% 

14.1% 

23.5% 

54.1% 

4 

Indirectly through understanding 
the grammar/vocabulary better 

10.6% 

25.9% 

38.8% 

24.8% 


Table 3. Relevance of factors/tools for improving listening skills 


The responses to this question indicate that exposure to language tutors and the online listening 
quizzes proved to be the most effective tools for teaching to listen. One student commented that the 
listening quizzes were useful in that they provided “an opportunity to listen to different native 
speakers aside from our own tutors”. Unsurprisingly, more than half of the students indicated that 
their own personal exposure was the least effective source for learning to listen. We cannot 
ascertain whether this was due to the lack of exposure or the inability to learn from other sources. 
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Impact of listening quizzes on students ’ overall learning 

In addition to evaluating the efficacy of the language quizzes for teaching how to listen, students 
were asked to indicate whether the listening quizzes had helped them improve their overall learning 
of Italian. Thirty-three percent responded, “yes, considerably”, 61% “yes, somewhat” and 6% 

“no”. Upon further investigation it was revealed that most of those who selected “no” also indicated 
that the quizzes did not help them improve their listening skills and that they did not enjoy the 
quizzes (indicating either “no” or “neutral” in all cases). Overall, these results therefore suggest that 
not only do students perceive these quizzes as effective in teaching to listen, but they also perceive 
them as helpful for their overall learning of Italian. Absalom and Rizzi (2008, p. 60), in their study 
on the comparison between the effects of online listening and online text-based tasks, concluded 
that “online listening tasks strongly promote an integrative and deep approach to learning”. Our 
study shows that students recognised the value of the quizzes in facilitating second-language 
learning as a whole, making students’ learning, in the words of Vandergrift and Goh (2012, p. 

89),’’‘more transferable to new situations”. 

Due to the anonymous nature of the responses elicited via the survey, it was not possible to 
correlate the students’ perceptions of the usefulness of the quizzes with a measure of success based 
on their course results. However, to verify if LC practice would lead to successful overall 
performance, we conducted some complementary analysis and matched the students’ results in the 
10 listening quizzes against their results in the final written exam. We tested this hypothesis with 
the ITAL1401 cohort (116 students; two students did not sit their final exam and were excluded 
from the count). The two-hour written exam included a variety of tasks such as cloze-tests, reading 
comprehension and semi-guided writing. 

Figure 12 shows the distribution of the students who achieved a high distinction (HD) average in 
the 10 LC tasks matched against their mark in the final exam. 
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Figure 12. Exam results and listening comprehension marks 
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Students with an HD in the listening tasks are concentrated in the band of those who achieved an 
HD for the exam. In fact, the analysis indicates that most of these students also achieved an HD in 
the final exam. Out of the 78 students who have achieved an HD in the listening comprehension, 
55%, or 43 students, scored n HD in their final exam. 8 

To further analyse the correlation between the data, the differences between the two assessment 
tasks were taken, and a histogram was plotted (Figure 13). The histogram shows that almost 50% of 
the students in the cohort display a small difference between their LC mark and their final exam 
mark. In other words, students generally maintained their grades between the two assessments. For 
example, if a student achieved a D score in their LC, the chances of them maintaining the same 
grade for their final exam was about 50%. Hence, from the two analyses presented here based on 
marks, there is a certain correlation between the students’ LC skills and their overall language 
proficiency, as assessed in the final exam. 


Histogram of Differences Between Listening Marks and Final 
Exam Marks 



Figure 13. Differences between students’ final exam mark and their average mark for the 
quizzes 


It must be considered, however, that other factors would have contributed to the students’ exam 
mark (e.g. general motivation, aptitude, regular grammar and vocabulary practice, study, use of 
learning strategies, etc.). In addition, it is unclear whether those students who displayed positive 
attitudes towards the online listening and declared that the listening materials were useful are the 
same students who achieved the highest marks in the exam. However, this complementary analysis 
provides another perspective on the understanding that systematic listening practice and progress 
are linked. 


6. Conclusions 

The outcome of this empirical study pertaining to students’ perceptions of the online assessment 
materials is overall very positive, and this is further supported by the students’ comments. Although 
our findings are based on self-report data rather than measures of successful performance, they 


8 While it is true that students had unlimited attempts for the quizzes, with the risk that they would not necessarily listen to 
the audio text every time they attempted the quiz, 93% of the students declared in the survey that they listened to the audio 
files more than once (cf. 5.1 above). 
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contribute to our understanding of effective listening pedagogy, and support the systematic 
incorporation of online listening tasks in the curriculum. From the teachers’ perspective, the 
listening quizzes improved the courses significantly and will be implemented in the future. They 
freed up valuable time for in-class work, provided effective tools for the assessment of listening 
skills and promoted learning motivation. Students seem to have appreciated the flexibility offered 
by the online environment and the effectiveness of the quizzes in developing their listening skills as 
well as their overall language proficiency. 

The integration of the quizzes was successful in terms of completion and engagement with the task, 
with 97% of the students completing between half to all of the quizzes. Ninety-three percent 
declared that they listened to the audio tracks more than once, and the majority also took the 
quizzes that did not count towards their final mark. The task was enjoyed by the majority (67%) of 
the students, and 66% expressed a preference for online assessment versus in-class assessment, in 
spite of the technical difficulties that some encountered. Most importantly, the vast majority (92%) 
answered that the quizzes helped them improve their listening skills, and 94% said the quizzes 
helped them with their overall learning of Italian. 

The blended learning experience offered to the students was successful due to a combination of 
factors. The delivery mode of the materials meant that students had access to the listening tasks in a 
flexible way, which maximised their opportunity for learning, as they could listen and listen again 
as often as they liked. Effective listening requires much listening practice and students were 
provided with a considerable amount of systematic work both online and in class. This gave them 
plenty of opportunity to put into practice bottom-up and top-down listening strategies, such as 
selective attention, activating prior knowledge to predict what they might hear and verifying. In 
addition, the online tasks proved to be enjoyable, which increased their motivation to learn and 
gave them higher confidence in persisting with the progressively more challenging tasks. We 
believe the online delivery instructed our students in self-efficacy and in having a sense of control 
over the listening process, which is fundamental for effective listening. The implications of this 
investigation are clear: listening practice and assessment can effectively be moved out of the 
classroom and into the digital space, provided it is grounded in sound pedagogical choices. In this 
sense, our work validates blended learning, and supports it not only as a mode of presenting the 
materials, but also as an effective way of assessing the students, within a holistic design of the 
learning experience that is aimed at progress and is student-centred. 

Further research on blended LC learning should consider how the use of authentic audio and video 
materials linked to online listening quizzes can also instil social and intercultural understanding in 
the learners; this area has not yet been fully explored. 

Suggestions for future improvements in the implementation of blended learning for listening via 
quizzes include: 1) gradually reduce the number of quiz attempts from unlimited to a set number, to 
discourage students from reattempting the quiz without listening to the audio track; 2) provide 
students with a transcript of the audio text, after a given number of quizzes have been submitted, for 
further verification and study of the input; this emerged from a student’s comment and from in- 
class discussion, which reflects their need to develop word-recognitions skills and awareness of the 
relationship between word and meaning; and 3) further expand the component of explicit 
instruction on LC strategies in the classroom context to further support blended learning. 
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